Automated implant movement analysis systems and related methods

ABSTRACT

Methods, systems, workstations, and computer program products that provide automated implant analysis of batches of image data sets of a plurality of different patients having an implant coupled to bone using a first data set of a first patient from the batch of image data sets, the first data set comprising a first image stack and a second image stack and allowing a user to select parameter settings for implant movement analysis of the implant including selecting a first object of interest and a second reference object. Measurements of movement of the implant and/or coupled bone can be automatically calculated and selected parameter settings can be automatically propogated to other image data sets of other patients of the batch of image data sets and measurements for the batch of image data sets of others of the different patients can be automatically calculated.

RELATED APPLICATIONS

This application claims the benefit of and priority to U.S. ProvisionalApplication Ser. No. 62/805,056, filed Feb. 13, 2019 and U.S.Provisional Application Ser. No. 62/824,598, filed Mar. 27, 2019, thecontents of which are hereby incorporated by reference as if recited infull herein.

FIELD OF THE INVENTION

The present invention is related to analysis of implanted medicaldevices.

BACKGROUND

Patients with pain or reduced function in their musculoskeletal system(bone/tendon/musculature) can sometimes be treated with artificialimplants, for instance joint replacement implants. These may or may notbe successfully anchored in the surrounding bone. If they are notsuccessfully anchored, they will eventually have to be replaced. Theearlier this is performed the better since the surrounding bone can bedestroyed (osteolysis) when adjacent to a loose implant. On the otherhand, if there is no loosening, the replacement surgery (in medicalterms: revision surgery) should be avoided since there is a patientsafety risk from complications such as infections, and also because itwould incur a substantial unnecessary cost.

However, to find out whether an implant is loose or unanchored ormigrating has conventionally been difficult or, at least, error-prone,since only far-progressed loosening has been possible to reliably detectnon-invasively. Thus, a surgeon makes the important decision as whetherto perform implant revision or not based on unclear information.

A new diagnostic process which aids greatly in solving this diagnosticproblem has been developed, by Sectra called Implant Movement Analysis(IMA). In IMA, two so-called provocation CT (Computed Tomography) scanscan be taken of the patient; in one CT the joint under investigation isbent (provoked) in one direction, in the other CT the joint is bent inanother direction (FIGS. 1A, 1B). The resulting CT images are thenprocessed in software performing a rigid registration and overlaying thetwo CT images (FIG. 1C) so that the implant is held still on the screenwhile the doctor flicks back and forward (FIGS. 1D, 1E) between the twoprovocation CT images. Any small movement (typically down to at least0.5 mm or 0.5 degrees) between implant and bone will be visible whenswitching between the two images, providing the requested earlyloosening detection.

The IMA method can also be used for other movement, migration or wearanalyses. One example is movement between different parts of one or moreimplant, where movement could indicate an implant breakage/malfunction.Another example is longitudinal analysis of migration/wear, applied totwo CT scans from different points in time (such as directly aftersurgery and months/years later) (FIGS. 2A-2C) instead of the provocationprocedure typically of images taken at one time point, described above.A registration of the implant and migration-relevant bone can begenerated with contours from different time points (FIG. 2D). Apart fromthe qualitative visual assessment described above, quantitative measureson the movements can be derived and presented as measures of migrationover time in terms of translation and rotation (FIG. 2E).

Another implant movement analysis method developed and sold by Sectra iscalled CT implant Micromotion Analysis (CTMA). CTMA is a quantitativeanalysis of change in location and rotation of an object between two CTstacks. The change is reported relative to a secondary object, referredto as the reference object. Thus, one comparison involves at least twoobjects each represented in at least two CT stacks. The typical scenariofor CTMA is to compare several time points, to see whether an implant ismigrating in the body over time.

The basic movement analysis process is the result from many years ofresearch from the Weidenhielm group at Karolinska Institute. Sectra hasin recent years created motion analysis products, seehttps://sectra.com/medical/product/sectra-ima/. and for CTMA, seehttps://sectra.com/medical/product/sectra-ctma/).

It is also noted that Radiostereometric Analysis (RSA) has been used forevaluating implanted devices over time. Thus, for some applications RSAcan be seen as an alternative to IMA and CTMA. However, RSA is mainlyused as a research tool, rather than a clinical tool, due to itscomplexity and cost and the fact that patients need to be given specialimplants if they are to be a part of an RSA study, something which isnot the case for patients included in IMA and CTMA studies.

Examples of articles describing implant analysis are listed in thisparagraph and incorporated by reference as if recited in full herein.Acetabular component migration in total hip arthroplasty using CT and asemiautomated program for volume merging, Olivecrona et al, ActaRadiologica, 2002. Stability of acetabular axis after total hiparthroplasty, repeatability using CT and a semiautomated program forvolume fusion, Olivecrona et al., Acta Radiologica, 2003. Assessing Wearof the Acetabular Cup Using Computed Tomography: an ex vivo Study,Olivecrona et al., Acta Radiologica, 2005. A new technique for diagnosisof acetabular cup loosening using computed tomography: Preliminaryexperience in 10 patients, Olivecrona et al., Acta Orthopaedica, 2008.Motion analysis of total cervical disc replacements using computedtomography: Preliminary experience with nine patients and a model,Svedmark et al., Acta Radiologica, 2011. Computed Tomography vs. DigitalRadiography Assessment for Detection of Osteolysis in AsymptomaticPatients With Uncemented Cups: A Proposal for a New ClassificationSystem Based on Computer Tomography, Sandgren et al., The Journal ofArthroplasty, 2013. A CT method for following patients with bothprosthetic replacement and implanted tantalum beads: preliminaryanalysis with a pelvic model and in seven patients, Olivecrona et al.,Journal of Orthopaedic Surgery and Research, 2016.

SUMMARY OF EMBODIMENTS OF THE INVENTION

Embodiments of the invention provide systems, methods and/or imageprocessing circuits that provide technological improvements to theconventional IMA/CTMA process which can make the systems more useful andreliable in clinical settings.

Embodiments of the invention provide technological improvements inautomated implant analysis systems that can result in efficiency, makinguser handling as well as novice training faster, and providing precisionin motion detection, increasing the quality of the decision support onpossible revision surgery or on assessment on implant function.

Embodiments of the invention can provide automated analysis systems thatcan generate relevant measurements of implants across batches of patientimages and provide quality assurance overview summaries to allow a userto review the analysis and confirm the measurements are done correctly.

Embodiments of the invention provide automated analysis systems andmethods that can connect CT image stacks to be compared; tune parametersfor segmenting bone and metallic implants in the images; and selectwhich segmented objects, or parts of objects, that are used asregistration and measurement targets.

Embodiments of the invention provide automated implant analysis methodsthat include: obtaining a batch of image data sets of a plurality ofdifferent patients having an implant coupled to bone; providing a firstdata set of a first patient from the batch of image data sets, the firstdata set comprising a first image stack and a second image stack;allowing a user to select parameter settings for implant movementanalysis of the implant including selecting a first object of interestand a second reference object; segmenting the first image stack and thesecond image stack to identify corresponding object pairs of the firstobject and the second object; registering each of the identified objectpairs; automatically calculating measurements of movement of the implantand/or coupled bone after the registration; automatically propagatingthe selected parameter settings to other image data sets of otherpatients of the batch of image data sets; and electronicallyautomatically repeating the segmentation, registration and calculatedmeasurements for the batch of image data sets of others of the differentpatients.

The first object can be a target study object. One of the first objectand the second reference object can be the implant. The parametersettings can include relating a coordinate system to the referenceobject, and identifying which measurements are to be calculated such asrotation and location of selected points of interest of the target studyobject.

After the first data set is analyzed, for identifying the first andsecond objects in the image data sets of the others of the differentpatients before a respective registration, the implant and associatedposition can first be automatically electronically identified, then thesecond object can be identified using the implant position as guidance.

The method can further include automatically electronically defining acohort analysis template based on the user selected parameter settingsand the first object and the second reference object of the data set ofthe first patient. The cohort analysis template can be used toautomatically propagate the selected parameter settings to the otherimage data sets thereby using identical parameter settings across allcomparisons provided by the calculated measurements.

The method can further include providing a display of results of thecalculated measurement of movement of the implant in the batch of imagedata sets.

The method can further include providing a visualization of anaggregated view of overlying registered images of image data sets of thedifferent patients with overlapping regions visually deemphasizedrelative to outliers.

The overlapping regions can have a reduced optical opacity relative tothe outliers or can be presented translucent or transparent.

The visualization can be presented with sub-regions shown with differentopacities or contrast. Different sub-regions are shown with an opacityand/or contrast that is inversely proportional to a number of objectsoverlapping in a respective sub-region.

The method can further include displaying thumbnail images of registeredobjects of different patients, optionally sorted by amount of calculatedmeasurement of movement.

The method can further include electronically linking thumbnail imagesto an aggregated view of all the registered objects of the differentpatients and allowing a user to navigate from a selected thumbnail imageto the aggregated view, optionally with the selected thumbnail imagevisually emphasized in the aggregated view relative to other registeredimages of other thumbnail images.

The segmenting step can be carried out automatically. The method canfurther include automatically repeating the segmenting step usingdifferent tuning parameters before the registering step to therebyprovide more accurate segmentation of the first and second objects.

The method can further include, before the segmenting step,automatically selecting relevant image stack pairs from the first andsecond patient image stacks. The image stack pairs can have the firstand/or second object.

The method can further include providing an electronic implant blueprintcorresponding to the implant. One or more of the segmenting, registeringor calculating measurements can be carried out using the electronicimplant blueprint.

The method can further include providing an electronic implant blueprintcorresponding to the implant. The segmenting can be carried out aplurality of times for the first data set using a plurality of differentthreshold levels that varies noise levels to match the blueprint withthe segmented first and/or second object.

The method can further include providing an electronic implant blueprintcorresponding to the implant. The registration can include matchingpoint clouds of points generated on one or more surfaces of the firstand/or second object.

The method can further include: providing an electronic implantblueprint corresponding to the implant; defining points on theelectronic implant blueprint where measurements are to be made; andtransferring the defined points to an image-domain implant. Theregistration can be carried out using the defined points.

The method can further include: providing an electronic implantblueprint corresponding to the implant; electronically definingreference points on the electronic implant blueprint; thenelectronically translating the blueprint reference points to thesegmented implant object. The automatically calculating measurements ofmovement of the implant and/or coupled bone after the registration canbe carried out using the translated blueprint reference points.

The method can further include: providing an electronic implantblueprint corresponding to the implant; and electronically definingfocus surface locations on the electronic implant blueprint. Before theregistration, the method can include automatically electronicallytranslating the blueprint focus surface locations to correspondinglocations on segmented first and/or second object, then generating anunevenly distributed point cloud with higher concentration at focussurface locations, then performing the registration using the generatedpoint cloud.

Before the registration, the method can include automaticallyelectronically deriving shape characteristics across one or moresurfaces of a segmented first and/or second object, then electronicallydefining high curvature locations as focus surface locations, thenelectronically generating an unevenly distributed point cloud withhigher concentration at focus surface locations. The registration can becarried out by electronically performing the registration using thegenerated point cloud.

The method can further include: providing an electronic implantblueprint corresponding to the implant in the first patient;electronically comparing a segmented first or second reference object tothe implant blueprint; and adjusting segmentation parameters andrepeating the segmentation of the first data set.

A workstation with an image processing circuit or in communication withan image processing circuit configured to carry out any of the methodsteps described herein.

Embodiments of the invention are directed to automated implant analysismethods that: obtain first and second sets of patient image stacks of apatient having at least one implant coupled to bone; segment bone and/orthe at least one implant in the first and second image stacks to definesegmented whole objects and/or segmented parts of objects; automaticallyselect which segmented whole objects and/or segmented parts of objectsto use as registration and measurement targets; register the selectedrelevant image stack pairs from the first and second patient imagestacks using the selected segmented whole objects and/or the segmentedparts of objects; and use the registered selected segmented wholeobjects and/or parts of objects to display or automatically measuremovement of the implant.

The segmenting can be carried out automatically.

The method can further include automatically repeating the segmentingstep using different tuning parameters before the registering to therebyprovide more accurate segmented whole objects and/or segmented parts ofobjects for the registration.

The segmenting bone and the at least one implant can be carried out aplurality of times using a plurality of different threshold levels thatvaries noise levels and amount of bone included in the defined segmentedwhole objects and/or the segmented parts of objects.

The method can further include, before the segmentation, automaticallyselecting relevant image stack pairs from the first and second patientimage stacks. The image stack pairs typically have at least one commontarget object or part of a target object for analysis therein.

The automatic selection can be carried out to select an entire implantas one of the segmented whole objects as a registration target for theregistration step.

The automatic selection of which segmented whole objects and/orsegmented part of objects to use as registration and measurement targetscan be carried out to select part of the implant as one of the segmentedparts of objects as a registration target for the registration.

The automatic selection of which segmented objects and/or segmented partof objects to use as registration and measurement targets can be carriedout by automatically electronically matching segmented whole and/orpartial bone objects to pre-defined templates of target objects.

The automatic selection of which segmented objects and/or segmented partof objects to use as registration and measurement targets can includeautomatically electronically matching segmented whole and/or parts ofimplant objects to pre-defined templates of target whole and/or parts ofobjects, optionally aided by segmented bone objects.

The automatic selection of which segmented objects and/or segmented partof objects to use as registration and measurement targets can includeautomatically electronically creating a set of different analysistargets for movement analysis based on the selected segmented wholeobjects and/or segmented parts of objects matched to pre-definedtemplates of target whole and/or partial objects.

The method can further include automatically grouping matched objectsaccording to pre-defined templates of target whole and/or partial objectgroups.

The method can further include automatically electronically creating aset of different analysis targets for movement analysis based onsegmented whole objects and/or segmented parts of objects matched topre-defined templates of target whole and/or partial objects.

The method can further include electronically removing or omitting apart of the anatomy of the patient in the relevant pairs of image stacksbefore the registration.

The method can further include, before the automatic selection,automatically removing or discarding non-relevant objects and/or partsof objects from a larger set of segmented objects and/or segmented partsof objects.

The method can further include providing an electronic implant blueprintcorresponding to the implant in the patient. One or more of thesegmentation, selection, registration or measurements can be carried outusing the electronic implant blueprint.

The method can further include: providing an electronic implantblueprint corresponding to the implant in the patient; electronicallydefining reference points on the electronic implant blueprint; thenelectronically translating the blueprint reference points to thesegmented implant object. The automatic calculation of measurements ofmovement of the implant and/or coupled bone after the registration canbe carried out using the translated blueprint reference points.

The method can further include providing an electronic implant blueprintcorresponding to the implant in the patient; and electronically definingfocus surface locations on the electronic implant blueprint. Before theregistration, the method can include automatically electronicallytranslating the blueprint focus surface locations to correspondinglocations on segmented implant objects and/or parts of objects, thengenerating unevenly distributed point cloud with higher concentration atfocus surface locations, then performing the registration using thegenerated point cloud.

The method can include, before the registration, automaticallyelectronically deriving shape characteristics across one or moresurfaces of a segmented implant object and/or a segmented part ofimplant object as one or more of the segmented whole objects and/orsegmented parts of objects, then electronically generating unevenlydistributed point cloud with concentration varying according to shapecurvature, then electronically performing the registration using thegenerated point cloud.

The method can include providing an electronic implant blueprintcorresponding to the implant in the patient; electronically comparing asegmented implant object to the implant blueprint; and adjustingsegmentation parameters and repeating the segmentation.

Other embodiments are directed to an automated implant analysis methodsthat includes: obtaining first and second sets of patient image stacksof a patient having at least one implant coupled to bone; automaticallyidentifying objects as relevant analysis targets, each analysis targetassociated with at least one relevant stack pair; automaticallyperforming movement analysis for each identified analysis target;electronically storing an analysis target set of different analysistargets and associated movement analysis results; allowing a user toselect an analysis target from the analysis target set; and displayingthe movement analysis result of the selected analysis target.

Yet other embodiments are directed to automated implant analysis methodsthat include: obtaining first and second sets of patient image stacks ofa patient having at least one implant coupled to bone; segmenting boneand/or the at least one implant in the first and second image stacks todefine segmented whole objects and/or segmented parts of objects;automatically electronically deriving shape characteristics for thesegmented whole objects and/or segmented parts of objects; automaticallyelectronically using the shape characteristics to calculate risk ofregistration errors; use the registration error risk for automatic ormanual selection of segmented whole objects and/or segmented parts ofobjects to use as registration and measurement targets; registering theselected relevant image stack pairs from the first and second patientimage stacks using the selected segmented whole objects and/or thesegmented parts of objects; and using the registered selected segmentedwhole objects and/or parts of objects to display or automaticallymeasure movement of the implant.

Still other embodiments are directed to automated orthopedic analysismethods that include: obtaining first and second sets of patient imagestacks of a patient; segmenting bone in the first and second imagestacks to define segmented whole objects and/or segmented parts ofobjects; automatically selecting which segmented whole objects and/orsegmented parts of objects to use as registration and measurementtargets; and registering the selected relevant image stack pairs fromthe first and second patient image stacks using the selected segmentedwhole objects and/or the segmented parts of objects; and using theregistered selected segmented whole objects and/or parts of objects todisplay or automatically measure movement of the bone.

Embodiments of the invention are directed to workstations that can beconfigured to carry out any of the methods, or portions thereof,described herein.

Embodiments of the invention are directed to image processing circuitsthat are configured to carry out any of the methods, or portionsthereof, described herein.

It is noted that any one or more aspects or features described withrespect to one embodiment may be incorporated in a different embodimentalthough not specifically described relative thereto. That is, allembodiments and/or features of any embodiment can be combined in any wayand/or combination. Applicant reserves the right to change anyoriginally filed claim or file any new claim accordingly, including theright to be able to amend any originally filed claim to depend fromand/or incorporate any feature of any other claim although notoriginally claimed in that manner. These and other objects and/oraspects of the present invention are explained in detail in thespecification set forth below.

BRIEF DESCRIPTION OF THE DRAWINGS

The patent or application file contains at least one drawing executed incolor. Copies of this patent or patent application publication withcolor drawings will be provided by the Office upon request and paymentof the necessary fee.

FIGS. 1A and 1B are schematic illustrations of examples of two images ofa patient with an implant acquired with different physical provocations(with the stacks symbolically represented by a single 2D slice).

FIG. 1C is an example of the two image stacks being registered targetingthe implant such that the implant is the same location when the imagesare overlaid.

FIGS. 1D and 1E are images of different blendings of the two imagestacks which can be shown serially and quickly to identify movement orsmall spatial differences of objects close to the implant.

FIGS. 2A-2C are image stacks of a patient with an implant acquired atdifferent time points (the stacks symbolically illustrated by a single2-D slice).

FIG. 2D is a visualization of a registration of the implant withcontours in different colors representing different time points of thedifferent images shown in FIGS. 2A-2C.

FIG. 2E is a report of quantitative measures of the migration over timein terms of translation and rotation and/or other movement according toembodiments of the present invention.

FIG. 3 is an example image stack according to embodiments of the presentinvention.

FIG. 4 is an example image stack pair according to embodiments of thepresent invention.

FIG. 5 is an example of objects in a respective image stack pairaccording to embodiments of the present invention.

FIG. 6 is an example implant blueprint (e.g., CAD drawing) of an implantunder analysis according to embodiments of the present invention.

FIG. 7 is a flow chart of an overview of a prior art manual movementanalysis method.

FIG. 8 is a flow chart of an automated movement analysis methodaccording to embodiments of the present invention.

FIG. 9 is a flow chart of another automated movement analysis methodaccording to embodiments of the present invention.

FIG. 10 is an example automated movement analysis system with modulesaccording to embodiments of the present invention.

FIG. 11 is an example of a flow chart of an automated stack pairselection system according to embodiments of the present invention.

FIG. 12 is an example flow chart of actions for automated thresholdingfor segmenting CT stacks according to embodiments of the presentinvention.

FIG. 13 is an example flow chart of actions for automated objectselection of segmented objects according to embodiments of the presentinvention.

FIG. 14 is a CT image with an example implant, sagittal slice through aCT stack showing larger spine implant with four pedicle screws that canbe analyzed by the automated implant movement systems and methodsaccording to embodiments of the present invention.

FIG. 15 shows a display with a plurality of thumbnail images of CTstacks of a patient with an implant that can be automatically pairedaccording to embodiments of the invention, where stack number S1 and S3are relevant to pair together.

FIG. 16A is an example image of a hip implant having been automaticallysegmented with the same implant in the pair of CT stacks according toembodiments of the present invention.

FIG. 16B is an example image of a spine implant having beenautomatically segmented with the same implant in the pair of CT stacksaccording to embodiments of the present invention.

FIG. 16C is an example image automatically segmented at a first definedthreshold with bone segmented at a high threshold value (not all boneincluded) according to embodiments of the present invention.

FIG. 16D is an example image automatically segmented at a second definedthreshold, with bone segmented at a lower threshold value than in FIG.16C (most bone included, also shown with some noise/non-bone) accordingto embodiments of the present invention.

FIG. 16E is an example image automatically segmented at a third definedthreshold, with bone segmented at a lower threshold value than in FIG.16D (all bone included, also shown with increased noise/non-bonerelative to FIGS. 16C and 16D) according to embodiments of the presentinvention.

FIGS. 17A-17J are images illustrating example automated registrationprocessing according to embodiments of the present invention.

FIG. 18 is a flow chart of an overview of a prior art manual CTMA batchanalysis method.

FIG. 19 is a flow chart of an automated batch analysis method accordingto embodiments of the present invention.

FIG. 20 is a flow chart of an automated batch analysis method that caninclude cohort propagation according to embodiments of the presentinvention.

FIG. 21 is a flow chart of a segmentation process that uses an implantblueprint according to embodiments of the present invention.

FIG. 22 is a flow chart of actions that can be carried out to improveand/or assess registration accuracy using surfaces of the implantblueprint according to embodiments of the present invention.

FIG. 23 is a flow chart of actions that can be carried out to improveand/or assess registration accuracy using the implant blueprint formeasurement according to embodiments of the present invention.

FIG. 24 is a flow chart of actions that can be carried out forperforming batch quality assurance according to embodiments of thepresent invention.

FIG. 25 is an image of an example view of a batch analysis system toinvestigate suspected CTMA processing errors which can be provided to auser for a quality assurance protocol according to embodiments of thepresent invention.

FIG. 26 is a schematic illustration of a system configured for automatedimage analysis according to embodiments of the present invention.

FIG. 27 is a schematic diagram of an example data processing systemaccording to embodiments of the present invention.

FIGS. 28A-28D are symbolic visualizations of an example reference pointon an implant blueprint translated to a corresponding segmented objectaccording to embodiments of the present invention.

FIGS. 29A-29C are symbolic visualizations of example point cloudgeneration useful for representing a target object according toembodiments of the present invention.

FIGS. 30A-30C are illustrations of example templates for measurementobjects according to embodiments of the present invention.

FIGS. 31A-31E are illustrations of an example target template collectionaccording to embodiments of the present invention.

FIGS. 32A-32C are example symbolic shapes of objects representingdiffering degrees of registration risk according to embodiments of thepresent invention.

FIGS. 33A-33C are example illustrations of a user interface providing anaggregated view of different patient images according to embodiments ofthe present invention.

FIGS. 34A-34D are illustrations of an example segmented implant can beused as guidance for segmentation of a related anatomical objectaccording to embodiments of the present invention.

FIGS. 35A-35C are illustrations of example measurement referencesaccording to embodiments of the present invention.

DETAILED DESCRIPTION OF EMBODIMENTS OF THE INVENTION

The present invention will now be described more fully hereinafter withreference to the accompanying drawings, in which some embodiments of theinvention are shown. This invention may, however, be embodied in manydifferent forms and should not be construed as limited to theembodiments set forth herein; rather, these embodiments are provided sothat this disclosure will be thorough and complete, and will fullyconvey the scope of the invention to those skilled in the art. Likenumbers refer to like elements throughout. It will be appreciated thatalthough discussed with respect to a certain embodiment, features oroperation of one embodiment can apply to others.

In the drawings, the thickness of lines, layers, features, componentsand/or regions may be exaggerated for clarity and broken lines (such asthose shown in circuit or flow diagrams) illustrate optional features oroperations, unless specified otherwise. The term “Fig.” (whether in allcapital letters or not) is used interchangeably with the word “Figure”as an abbreviation thereof in the specification and drawings. Inaddition, the sequence of operations (or steps) is not limited to theorder presented in the claims unless specifically indicated otherwise.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a,” “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, steps, operations, elements, and/orcomponents, but do not preclude the presence or addition of one or moreother features, steps, operations, elements, components, and/or groupsthereof. As used herein, the term “and/or” includes any and allcombinations of one or more of the associated listed items. Like numbersrefer to like elements throughout. In the figures, the thickness ofcertain lines, layers, components, elements or features may beexaggerated for clarity. As used herein, phrases such as “between X andY” and “between about X and Y” should be interpreted to include X and Y.As used herein, phrases such as “between about X and Y” mean “betweenabout X and about Y.” As used herein, phrases such as “from about X toY” mean “from about X to about Y.”

Unless otherwise defined, all terms (including technical and scientificterms) used herein have the same meaning as commonly understood by oneof ordinary skill in the art to which this invention belongs. It will befurther understood that terms, such as those defined in commonly useddictionaries, should be interpreted as having a meaning that isconsistent with their meaning in the context of the specification andrelevant art and should not be interpreted in an idealized or overlyformal sense unless expressly so defined herein. Well-known functions orconstructions may not be described in detail for brevity and/or clarity.

It will be understood that when a feature, such as a layer, region orsubstrate, is referred to as being “on” another feature or element, itcan be directly on the other element or intervening elements may also bepresent. In contrast, when an element is referred to as being “directlyon” another feature or element, there are no intervening elementspresent. It will also be understood that, when a feature or element isreferred to as being “connected” or “coupled” to another feature orelement, it can be directly connected to the other element orintervening elements may be present. In contrast, when a feature orelement is referred to as being “directly connected” or “directlycoupled” to another element, there are no intervening elements present.The phrase “in communication with” refers to direct and indirectcommunication. Although described or shown with respect to oneembodiment, the features so described or shown can apply to otherembodiments.

The terms “circuit” and “module” are used interchangeably and refer tosoftware embodiments or embodiments combining software and hardwareaspects, features and/or components, including, for example, at leastone processor and software associated therewith embedded therein and/orexecutable by the at least one processor and/or one or more ApplicationSpecific Integrated Circuits (ASICs), for programmatically directingand/or performing certain described actions, operations or method steps.The circuit or module can reside in one location or multiple locations,it may be integrated into one component or may be distributed, e.g., itmay reside entirely in a workstation or single computer, partially inone workstation, cabinet, computer, or server and/or totally in a remotelocation away from a local display at a workstation. The circuit ormodule can communicate with a local display, computer and/or processor,over a LAN, WAN and/or internet to transmit images or analysis results.

The term “automatically” means that the operation can be substantially,and optionally entirely, carried out without human or manual input, andis typically programmatically directed and/or carried out. The term“electronically” includes both wireless and wired connections betweencomponents. The term “programmatically” means that the operation or stepcan be directed and/or carried out by a (digital signal) processorand/or computer program code. Similarly, the term “electronically” meansthat the step or operation can be carried out in an automated mannerusing electronic components rather than manually or using merely mentalsteps.

The term “clinician” refers to a physician or other personnel desiringto review medical data of a subject, which is typically a live human oranimal patient.

The term “user” refers to a person, or device associated with thatperson, that uses the noted feature or component, such as a technician,orthopedic doctor or other clinician, researcher or expert.

The term “about” means that the recited parameter can vary from thenoted value, typically by +/−20%.

The term “PACS” refers to PICTURE ARCHIVING AND

Communication System.

The term “magnification” means the image resolution measured inmicrometers per pixel, applicable both for the scanned image and theimages displayed on screen. Higher magnification corresponds to a lowermicrometer per pixel value than lower magnification and vice versa.

The term “semi-automated” refers to an image processing system, method,module or circuit that employs user (e.g., orthopaedic doctor) input toperform certain functions such as one or more of: initiate an analysis,select an implanted implant of interest, or review results of a singlepatient movement analysis or batch movement analysis of aggregate orindividual analysis generated by automated systems.

As is well known to those of skill in the art, the term “registration”and/or “register” refers to an electronic process that aligns two ormore images taken at different times from common or different imagingequipment and/or sensors, typically from different orientations and/orangles, to geometrically align the images and/or objects or features inthe images for analysis.

Generally stated, the present invention provides technologicalimprovements to conventional manual-based systems. The registrationprocess can be automated in a manner that allows most of the problemsassociated with manual-based systems to be side-stepped or avoided. Forexample, embodiments of the present invention may provide automatedanalysis with fewer mistakes than a manual process. Techniques andsystems described herein may increase precision and/or reduce mistakesby avoiding suboptimal, inconsistent, and/or incorrect segmentationparameters. Similarly, techniques and systems described herein mayincrease precision and/or reduce mistakes by avoiding a suboptimal,inconsistent, and/or incorrect definition of registration and/ormeasurement targets. Techniques and systems described herein may utilizean implant blueprint to increase accuracy and precision in implantmovement measurement. Unlike manual processes, techniques and systemsdescribed herein may generate point clouds for representations ofimplants, which may increase precision in registration and thereby inmovement measurement. Each automation step can be applied in isolationor in any combination.

Referring to FIGS. 3-6, examination inputs are provided to the automatedimage analysis system 10. The inputs include image stacks 100 which canbe analyzed to identify stack pairs 100 p and relevant objects in therelevant image stacks 105 can be automatically identified. The term“image stacks” refers to slices of images of a patient that can beprovided in stacks as is well known to those of skill in the art. Theobjects 105 that can be identified include target anatomy and implants.The term “stack pairs” refers to a respective slice from one image thatcorrelates to a respective corresponding slice from a second image toprovide the stack pairs.

FIG. 6 illustrates that an implant blueprint 35 can also be provided asan input useful for movement analysis, registration and/or segmentation.The term “implant blueprint” refers to a digital model of an actualphysical implant (often referred to as a computer-aided design “CAD”model), in 3D or 2D that can be a scaled image or CAD drawing withdimensional and shape data. As will be discussed further below, theimplant blueprint 35 can comprise one or more defined reference points Rand/or focus surface locations F that can be used for one or more ofsegmentation, registration or measurements according to embodiments ofthe present invention.

The automated image analysis system 10 may display the images 100, imagestack pairs 100 p and/or objects 105 in the image stacks or image stackpairs. The system 10 can perform a registration with one or moretargeted implants, i.e., rotating and translating one of the stacks suchthat the implant is in the same location in the different image stackswhen images are overlaid.

The system 10 can analyze CT image stacks that are acquired based onprovocation or loading of a patient (FIGS. 1A, 1B) or CT image stacksacquired at several time points, typically months apart as shown inFIGS. 2A-2C. The stacks are symbolically represented by a single 2Dslice. In this example, the implant stem can be a target object 105(FIG. 5) that is analyzed and it is migrating down the bone (the bone ismoving upwards). The system 10 can perform a registration of the implantand migration-relevant bone and provide a visualization that canidentify movement of the implant shown with migration relevant bone(blue), optionally with contours of the migration relevant bone indifferent colors (i.e., green, orange and red) representing the timepoints of the implant in the different images. The system 10 can derivequantitative measures of the migration over time, in terms oftranslation and rotation, and present the results to a display 20.

While particularly suitable for analyzing CT images of CT image stacks,MRI images may also be evaluated by the automated system 10. Thus, whileembodiments of the invention will be discussed with respect to CTstacks, MRI image stacks or slices may also be used.

FIG. 7 illustrates an example overview of a prior art manual movementanalysis system. Note the number of required user inputs. Each IMAprocess currently typically requires between 5-10 minutes of time toallow for manual settings and PC calculation time. In addition, thesoftware requires relatively intensive training of 20+ hours of trainingfor a new user (typically a radiologist or orthopaedic surgeon) to getto a proficient level at producing informative visualizations. Evenafter full training, the efficiency of the users in doing manual stepscan vary. FIG. 8 is an overview of an example method provided by anautomated movement analysis system 10. A user can select a patient caseand request or initiate an automated analysis (block 200). Relevantpairs of CT stacks can be automatically identified (block 205). Objectsin the identified CT stacks can be automatically segmented (block 210).Segmentation is well known to those of skill in the art. See, e.g.,Digital Image Processing, Bernd Jähne, ISBN 3-540-67754-2,Springer-Verlag Berlin Heidelberg New York, the contents of which ishereby incorporated by reference as if recited in full herein.

Risk of error in subsequent registration of segmented objects can beautomatically derived based on shape characteristics of the segmentedobjects (block 212).

For example, the example method can provide the user with conditionnumbers, numbers which tell the user if it's likely a good registrationcan be achieved given the object's shape. For example, a sphere isassociated with a poor condition number for rotation, while a shape thatis asymmetrical in all dimensions can have a better condition number.FIGS. 32A-32C illustrate example target objects which will result indifferent condition numbers. FIG. 32A shows an object with a high riskof rotational registration error because of symmetry in all dimensions.An object with a high risk or rotational registration error around oneaxis (shown as a vertical axis in FIG. 32B) but low risk in other axescan be identified and this object may have a better condition numberthan the object shown in FIG. 32A. An object with a lower or lowest riskof registration error due to asymmetry in all dimensions can also beidentified (FIG. 32C). These condition numbers can be generated duringthe automatic registration analysis and provided to a user (block 212).The condition numbers can be in a range of 0-10, 0-100 or other ranges,typically with the high end, such as “10”, being associated with a highprobability of good registration and a low end, such as “0”, associatedwith a very low probability of good registration. The condition numberscan be a single number (shown as “0”, “5” and “10” for FIGS. 32A, 32B,32C, respectively) or provided as a set of numbers, one for each of thethree axes, X,Y,Z (shown as “[0, 0, 0]”, “[1, 9, 9]” and [“10, 10, 10”]for FIGS. 32A, 32B, 32C, respectively). Moreover, different conditionnumbers can be given for different parts of an object, such as a targetimplant object.

Objects in the segmented identified stacks can be automaticallyidentified as relevant analysis targets (block 215). Potential analysistargets, each target being in a CT stack pair, typically with one or two(or more) objects for each target, can be automatically presented,typically on a display (block 220). A user can be allowed to select ananalysis target from the potential analysis targets presented (block225). The automated presentation can be via thumbnail images and theselection can be by touch screen input, mouse click or other user input.The computed analysis for selected targets can be carried out theresults generated, optionally to a display screen (block 230). Theanalysis results can be provided to a user. A user can review theanalysis results (block 235).

FIG. 9 is another example method that can be provided by an automatedmovement analysis system 10 according to embodiments of the presentinvention. A patient case can be imported or otherwise provided to thesystem (block 250). The case can be automatically recognized as relevantfor movement analysis (block 255). Relevant pairs of CT stacks can beautomatically identified (block 260). Objects in the identified pairs ofCT stacks can be automatically segmented (block 265). As discussedabove, a risk of error in subsequent registration of segmented objectscan be automatically derived based on shape characteristics of thesegmented objects (block 212). Objects relevant as analysis targets canbe automatically identified (block 270). Potential analysis targets,each target being a CT stack pair with one or two objects each can bepresented. Potential analysis targets, each target being in a CT stackpair, typically with one or two (or more) objects for each target, canbe automatically identified (block 275). All potential analysis targetsidentified can be analyzed automatically to provide a set ofpre-computed analysis results for the patient case (block 280). A usercan be allowed to access the patient case and select an analysis targetfrom one of the identified and analyzed potential analysis targets(block 285). The user can review the pre-computed analysis results forthe selected target (block 290).

FIG. 10 is an example system 10 with a plurality of modules including astack pair identification module 300, a bone and implant segmentationmodule 310, a movement analysis module 320, a GUI module 330 andoptional patient case import module 340.

FIG. 11 illustrates that the system 10 can have an input 342 whereby auser selects a patient case and requests analysis and/or the patientcase import module (block 340) identifies examination as relevant toinitiate the analysis by the stack pair module (block 300).

The system 10, typically via the stack pair identification module 300,can be configured to retrieve meta-data for stacks in examination (block301). Relevant stacks can be identified through rule-based logic (block302R) and/or through a predictive artificial intelligence model (block302AI).

In some embodiments, stack identification based on rule-based logic mayutilize meta-data associated with the stacks, which may, for example, beelements in the DICOM standard format. Rules may be expressed as logicalconditions, such as “if element A is equal to X and element B is equalto Y, the stack is suitable for processing.” Rule-based pairing ofstacks may also be based on conditions that certain elements are equalor similar between the two stacks.

In some embodiments, stack relevance and pairing may be done throughartificial intelligence methods using meta-data such as, for example,DICOM elements. This could be done through document similarity methodsbased on vector space models, see, e.g., “Information Retrieval usingCosine and Jaccard Similarity Measures in Vector Space Model,” Jain etal., International Journal of Computer Applications, Vol 164, April2017, and “On modeling of information retrieval concepts in vectorspaces,” Wong et al., ACM Transactions on Database Systems, 1987, thecontents of which are hereby incorporated by reference as if recited infull herein.

In some embodiments, an artificial intelligence model identifyingrelevant stacks may be based on predicting anatomic content from theimage data, isolated from or in combination with the meta-data approach.Example methods for this purpose can be found in “CT scan rangeestimation using multiple body parts detection: let PACS learn the CTimage content,” Wang and Lundström, International Journal of ComputerAssisted Radiology and Surgery, Vol 11, 2016, Springer BerlinHeidelberg, and “A survey on deep learning in medical image analysis,”Litjens et al., Medical Image Analysis, Vol 42, December 2017, Elsevier,the contents of which are hereby incorporated by reference as if recitedin full herein.

The system 10 can discard or mark irrelevant stacks to excludeirrelevant stacks from further processing (block 304). The system 10 canidentify stack pairs through rule-based logic (block 305R) and/orthrough a predictive artificial intelligence model (block 305AI). Thesystem 10 can pass on CT stack pairs for user handling and/or furtherprocessing (block 307). For example, the system can display theidentified stack pairs in a display associated with a GUI.

FIG. 12 illustrates an example thresholding protocol that can be carriedout by the system 10, optionally using the bone and implant segmentationmodule 310. A CT stack pair is identified by the system (block 305). Thesystem 19 automatically performs initial segmentation performed usingpre-defined parameters (block 311). The system 10 automaticallyevaluates segmentation result for both CT stacks (block 312). The system10 automatically discards irrelevant objects from each set of segmentedobjects (block 313). The system automatically adjusts segmentationparameters (block 314) using rule-based logic (block 314R) and/or anartificial intelligence model (block 314AI). The system 10 automaticallyiterates segmentation with updated (adjusted) segmentation parameters(block 316).

In some embodiments, the initial segmentation can be done using modernmachine learning approaches, such as those described in “A survey ondeep learning in medical image analysis,” Litjens et al., Medical ImageAnalysis, Vol 42, December 2017, Elsevier the contents of which arehereby incorporated by reference as if recited in full herein.

In some embodiments, the adjustment of segmentation parameters can beperformed based on standard image processing techniques such as, forexample, region growing (see W. K. Pratt, “Digital Image Processing 4thEdition”, John Wiley & Sons, Inc., Los Altos, Calif., 2007) employing aHounsfield value threshold the contents of which are hereby incorporatedby reference as if recited in full herein.

In some embodiments, one option for such an adapted region growingapproach may be, given the initial segmentation and a threshold value,to expand the segmented area to include neighboring voxels withHounsfield values above the threshold and to shrink to exclude valuesbelow the threshold. The threshold Hounsfield value to use for thesegmentation adjustment may be derived through a regression approachusing an artificial intelligence model. The artificial intelligencemodel may be a convolutional neural network trained with image datahaving a best threshold value defined by human experts. After thesegmentation adjustment, additional post-processing to smooth theresulting shape can be applied, for instance using morphologicaloperations. Though segmentation adjustment utilizing region growing isprovided as an example, embodiments of the present invention are notlimited to this technique.

The system 10 can automatically pass on segmentation for user handlingand/or further processing (block 319).

FIG. 13 illustrates an example object selection protocol that can becarried out by the system 10, optionally using an object selectionmodule 350. The system 10 or a user can identify segmented objects, eacha paired object in the other stack of the CT stack air (block 351).

The system 10 can automatically match segmented bone objects topre-defined templates for target objects (block 352). The system 10 canautomatically match segmented implant objects to pre-defined templatesfor target objects (block 354), optionally aided by segmented boneobjects (block 355). The system can automatically group matched objectsaccording to pre-defined templates for target object groups (block 356).The system 10 can automatically create sets of alternative analysistargets based on matched objects (block 357). The system 10 can sortanalysis targets according to pre-defined criteria (block 358). Thesystem can provide computed movement analysis results for the sets ofanalysis targets and a user can select an analysis target from the setsof analyzed targets and the system can provide pre-computed analysisresults (blocks 285, 290, FIG. 9).

An example workflow is as follows:

-   1. Automated selection of CT stack pairs.    -   When a study is sent for movement analysis (by either automatic        or manual routing), a computer algorithm sorts through the CT        stacks for that examination (there may be many stacks in one        exam).    -   The algorithm couples each stack pair that is relevant for        movement analysis.    -   For instance, relevance can mean that provocation has been done,        that the same CT acquisition parameters (such as reconstruction        kernel) have been used, and that the time period between the two        scans are relevant (1-30 min for IMA, a set number of months for        CTMA).    -   The coupling can be done by rule-based logic or with Artificial        Intelligence trained to look at stack meta-data (DICOM        information) and/or image data.-   2. Automated algorithm for thresholding of bone and metal.    -   The CT stack pairs from the previous step are sent to an        algorithm which automatically performs segmentation of objects        consisting of relevant materials, typically metal and bone.    -   The segmentation can be done in many ways, where a typical        method component is thresholding, using ranges of CT voxel        values (Hounsfield values) for the materials.    -   The segmentation can automatically adjust the thresholding to        best fit the CT stack pair in question. This can, for instance,        be achieved through an artificial intelligence algorithm trained        to select the thresholds based on the image content, mimicking a        gold standard result that an expert would accomplish.    -   The segmentation can be performed in parallel for both CT stacks        in an image stack pair, to ensure that results are valid for        both stacks and that the results are comparable.    -   The set of segmented objects can be filtered. Reasons to remove        objects from further scrutiny could be that they are too small        fragments to be relevant, that they reside in medically        irrelevant locations, or that there is not a good match in size        and shape to an object in the other CT stack in the pair.-   3. Automated identification of target objects for analysis.-   The input to this step is that there are a number of segmented    objects, each object existing in both CT stacks of a stack pair. In    a manual workflow, the user would need to select analysis targets by    clicking in the image view.    -   According to embodiments of the invention, a list of possible        analysis targets is automatically created. A target comprises of        an object used for registration, and, optionally (in the        quantitative setting), another object used for measurements.    -   According to predefined analysis target types, the invention        identifies which objects that correspond to the sought        predefined targets.    -   The identification of objects can be done by first identifying        those corresponding to bone anatomy, and then identifying metal        components aided by their position relative to anatomical        objects (for instance, a metal object in close proximity to a        pelvis is more likely to be a hip prosthesis cup than a shoulder        implant).    -   The object categorization can also include merging objects which        should be treated as one object, for example two screws inserted        in the same vertebra.-   4. Automated initiation of analyses.    -   The analyses for which relevant objects were found in the        previous step are automatically initiated and their results are        stored.    -   The list of targets is presented to the user, who can review its        results by selecting an item in the list.    -   The analysis items can be presented as thumbnail views of images        with the respective objects highlighted. The list can be sorted        in order of relevance for the CT stacks in question.-   When the above functions are in place, with substantially all    necessary processing done automatically beforehand and stored as    pre-computed analysis results for different target analysis sets,    the system can be configured to allow an end user to have select    inputs as follows.    -   Select a patient case    -   Initiate the analysis, for instance by a button/menu option        connected to either of the two CT stacks, to the entire study,        or to the patient    -   Select the object(s) to analyze        -   May be carried out through symbolic representations in a            list, or by clicking the objects in the CT stack        -   For qualitative analysis, a single object can be sufficient            (the registration target), for quantitative analysis at            least one other, i.e., second reference object is also            needed (the measurement target)    -   Review the registration results        -   Adjustment options are available in case the automated            procedure is not sufficient-   Thus, embodiments of the invention can provide higher efficiency and    precision over conventional systems in performing this type of    implant movement analysis.

FIG. 14 is an image of a CT implant example. This is a sagittal slicethrough a CT stack, showing larger spine implant with four pediclescrews.

FIG. 15 illustrates a plurality of image stacks for stack selection,with stacks numbers Si and S3 relevant to pair.

FIGS. 16A-16E illustrate example segmentations.

FIG. 16A is an example CT stack pair with a hip implant shown as 3Drenderings, the top stack with the patient's leg provoked inwards andthe bottom stack with the patient's leg provoked outwards, and with thehip implant having been segmented (same patient and implant in bothstacks).

FIG. 16B is an example CT stack pair with a spine implant shown as 3Drenderings, the top and bottom stack with different provocations of thespine, respectively, and with the spine implant having been segmented(same patient and implant in both stacks).

FIGS. 16C-16E are example images from different thresholding levelsdefined in the Hounsfield unit scale of the CT image data. FIG. 16Ccorresponds to bone having been segmented at high value threshold (notall bone included). FIG. 16D corresponds to a lower threshold than thefirst, higher threshold (most bone included, but also somenoise/non-bone).

FIG. 16E is an image at a third segmentation threshold, further lowerthreshold (all bone included, substantial noise/non-bone).

FIGS. 17A-17J illustrate example images of the registration processaccording to embodiments of the present invention. FIG. 17A is a jointview of two CT stacks showing the segmented hip implant beforeregistration (same implant from paired CT stacks).

FIG. 17B shows that a part of the implant (metal rings of prosthesiscup) has been selected as a registration target, the same part selectedin both stacks, shown in different colors.

FIG. 17C shows the result of the registration: the matching of the metalrings is complete, color coding in appended scale and on the metal ringsrefers to spatial distance between object, i.e. registration accuracy,indicating accuracy is below 1.0 mm, primarily between 0.15 mm and 0.00mm.

FIG. 17D is an image of an alternative registration target, the full hipimplant is selected as registration target.

FIG. 17E is an image of another alternative registration target, a clipbox can be used to select only stem part of hip implant.

FIG. 17F is an image of a registered result for the target in FIG. 17E:the matching of the stem part is complete, color coded accuracy withcolor coding in appended scale and stem shown primarily below 1.0 mm andmostly in the 0.00 mm-0.15 mm range.

FIG. 17G is an image of another registration target, a bone target: aclip box has been used to select part of pelvis and femur asregistration target, only one of the two stacks shown.

FIG. 17H is an image of the registration result of the bone target inFIG. 17G after also removing the femur from the registration target: theregistration is complete, color coded accuracy with color coding inappended scale and stem primarily less than 1.0 mm, mostly between 0.00mm and 0.15 mm (femur moves between the two stacks and should notmatch).

FIG. 17I is an image of another registration target, a spine implant:the image show the resulting registration for only the lower pair ofpedicle screws in larger spine implant, color coded accuracy with colorcoding in appended scale and stem primarily less than 1.0 mm, mostlybetween 0.00 mm and 0.5 mm.

FIG. 17J is an image of another registration target of the spineimplant: registration carried out for only the second lowest pair ofpedicle screws in larger spine implant, color coded accuracy with colorcoding in appended scale and stem primarily less than 1.0 mm, mostlybetween 0.00 mm and 0.5 mm.

FIG. 18 illustrates an example overview of a prior art manual CTMA batchanalysis system. Note the number of required user inputs and that eachCT stack comparison is done manually from scratch.

FIG. 19 is an overview of another embodiment of an example implantanalysis system 10′ which can be configured to evaluate batches ofdifferent patient image stacks. This embodiment operates in a batch modeof running CTMA comparisons across an entire cohort. The analysis system10′ can be configured to allow for a semi-automated setting where theuser is only required to specify at one input the analysis to be madeand the parameter settings to be used across the entire cohort.

As shown, the system 10′ can be configured to import or allow a user toselect a batch of different patient cases for evaluation (block 400).

This embodiment may be particularly suitable for research studies usingCTMA across a patient cohort, typically to evaluate the performance of aspecific implant type or drug effect on implant performance. CTMA can beused to make one to four comparisons within a patient (thus, across twoto five time points), and repeat this for different patients within astudy, such as 10 to 50 patients within a research study. Larger cohortsand more comparisons are also possible.

Within a study, the analysis setup will typically be very standardized,such as comparison target, implant type and scanner parameters. Thiscontrasts with the scenarios targeted by patient-specific analysisdiscussed above where the setup of the individual comparison istime-consuming handling and the automated analysis systems 10 addressesthis problem.

For batch analysis, it can be important that parameter settings of theanalysis are identical across all comparisons. Unfortunately, makingthese settings manually as was done in the past is a time-consumingprocess which also increases the risk for random errors.

Embodiments of the invention provide an automated batch analysis system10′ that is configured to allow quality control to make sure theautomated measurements are done correctly, since the data sets are oftennoisy, and'it can be important that substantial errors in the analysesbe identified and corrected.

As also shown in FIG. 19, the system 10′ can be configured to select orallow a user to enter parameters for CTMA analysis (block 405). Theseentries can be in relation to an implant blueprint (block 405 b). Again,although described as used with CT images for CTMA analysis, otherdepiction equipment may be used in lieu of CT images.

The parameters selected or used in relation to implant blueprint caninclude implant blueprint data corresponding to a physical shape anddimensions and implant type of a target implant for review, optionallyidentifiable via product number, manufacturer, part number, product namein a list of options and/or can be provided via a GUI of visualizationsof different implants 35 for selection to define targets for analysis inthe image stacks.

The system 10′ can select the first patient case for analysis (block407), which can be in a study ID order, random order, by date,alphabetic order by last name or other order. Alternatively, a user canselect which case is the first patient case for analysis or which orderis preferred (block 407). The system identifies the relevant pair of CTstacks from the first patient case (block 409).

The system identifies the relevant pair of CT stacks from the firstpatient case (block 409).

The system 10′ initiates segmentation of the two relevant object pairsin the identified pair of CT stacks (block 411).

The system 10′ computes segmentation (block 412). The system canoptionally compute segmentation aided by data of the implant blueprint(block 412 b).

The system 10′ can initiate CTMA analysis (block 414). The system canperform registration for each of the two object pairs (block 416).

The system 10′ can perform registration for each of the two object pairsaided by data from the implant blueprint (block 416 b).

The system 10′ can compute the movement analysis for the registeredobjects (block 418).

The system 10′ can select or otherwise move to the next patient case foranalysis (block 420).

The steps 409-418 can be repeated until the end of the batch is reached.The system 10′ can be configured to allow a user to perform qualityassurance of the results of the entire batch (block 425).

FIG. 20 is an example of a batch analysis system 10′ with a cohortpropagation aspect according to embodiments of the present invention.Similar to FIG. 19, the system 10′ can be configured to import or allowa user to select a batch of different patient cases for evaluation(block 400). The system 10′ can be configured to select or allow a userto enter parameters for MA analysis (block 405). The system 10′ canselect the first patient case for analysis (block 407), which can be ina random order, by date, alphabetic order by last name or other order.Alternatively, a user can select which case is the first patient casefor analysis or which order is preferred. The system 10′ identifies theanalysis target and performs the movement analysis (block 414). Thesystem 10′ selects the next patient case (block 420) and repeats theanalysis (block 414) until the end of the batch. The system 10′ can beconfigured to allow a user to perform quality assurance of the resultsof the entire batch (block 425).

The batch analysis system 10′ can be configured to automaticallypropagate settings from one data set to the full cohort. The batchanalysis setup can allow a user or the system to specify, provide orobtain the following information: the object to study and the referenceobject (one of which is the implant), how the coordinate system relatesto the reference object, segmentation parameters such as valuethresholds, and what measurements that are to be reported (rotation andlocation of which points of the object of study). A user can make thesespecifications for a first data set and then the analysis system 10′ canpropagate these settings to all other data sets in the cohort.

A technical challenge is to automatically find the two objectsidentified in the first data set in the subsequent data sets. The batchanalysis system 10′ can address this challenge through registrationtechniques. For example, for each subsequent data set, a preferredprocess is to first identify the implant (as this is typically verysimilar between data sets) and as a second step identify the secondobject using the implant position as guidance. The implantidentification can employ the same registration technique acrosspatients that CTMA uses for registering the implant between time pointswithin a patient case. This can be carried out similar to the automaticidentification of target objects discussed above for thepatient-specific implant analysis system 10, but with a first data setdefining a batch analysis template instead of a generic template.

In most cases, the progagation corresponds to re-applying segmentation,registration, measurement etc as defined for the first patient. Butsometimes it could require a different processing step. For instance, anobject segmentation of the first patient may be initialized by the userclicking a seed point, whereas the corresponding object segmentation inthe rest of the batch is initialized based on registering the shape ofthe segmented object of the first patient with the object to segment,similar to how the blueprint is used in FIG. 28.

Another type of difference in propagating from first patient to othersis that the threshold value for segmentation may be set manually for thefirst patient, and automatically for subsequent patients, such that theshape and size of the object matches the first patient segmentation,e.g., automatically repeating the segmenting step using different tuningparameters before the registration to thereby provide more accuratesegmentation of the first and second objects.

The batch analysis system 10′ can provide greater efficiency andprecision in performing this type of implant movement analysis studiesover conventional systems and methods.

FIG. 21 illustrates an example of actions for segmentation using implantblueprint data for image (implant) analysis systems 10, 10′ according toembodiments of the present invention. The digital implant blueprint canbe used to improve segmentation to inform the segmentation of thecorresponding object in the CT stack. Generally stated, one way toutilize the implant blueprint is to automatically adjust thesegmentation parameters, such as threshold levels, to achieve an asclose as possible match between the entire device or a sub-portion ofthe device in the image blueprint and the segmented object.

Referring to FIG. 21, pre-defined, initial segmentation parameters canbe set or selected (block 500). The system then performs segmentation(block 502). The system 10, 10′ compares the segmentation result to theshape of the implant blueprint (block 505). If the shape or a boundaryof the object does not correspond to the implant blueprint, the system10, 10′ can adjust segmentation parameters (block 508), then repeat thesegmentation (block 502). The segmentation is complete when the shape ofthe implant blueprint is within the defined threshold. The resultingsegmentation can be used as input to further processing (block 510).

FIGS. 34A-34D illustrate an example of how a segmented implant I can beused as guidance for segmentation of a related anatomical object A.These illustrations are of an example hip implant in two-dimensions forease of discussion but typical segmentations are in three-dimensions.FIG. 34A illustrates an image with a segmented hip implant I. FIG. 34Billustrates a plurality of seed points Sp for femur segmentation, asdefined by their spatial relation to the implant I. FIG. 34C illustratesthat some seed points Sp are retained S_(R) while others are discardedor not used S_(D), based on their Hounsfield value. FIG. 34D illustratesa resulting segmentation of the anatomical object (femur) based on theretained seed points S_(R).

FIG. 22 illustrates an example of actions for registration using implantblueprint data for implant analysis systems 10, 10′ according toembodiments of the present invention. A digital implant blueprint can beutilized to improve registration. For example, registration can beperformed by a matching of point clouds, where the points are generatedon the surface of the object. Precision can be improved by making surethat the point cloud is more densely populated at sharp points or edgesof the object, such as the tip of the stem of a hip prosthesis. Anotherpossibility to utilize the image blueprint is to define such “sweetspots” for the point cloud generation, to be applied across the entirecohort

The term “ray casting” is well known to those of skill in the art andrefers to electronically casting rays to sample volumetric data sets tosolve a variety of problems in computer graphics and computationalgeometry. The term “point cloud” refers to a set of points distributedin a volumetric space to identify an object in that volumetric space,such as an implant or part of an implant and/or bone bounding avolumetric space. See, by way of example only, Dodin, P.,Martel-Pelletier, J., Pelletier, J.-P., Abram, F. (2011) A fullyautomated human knee 3D MRI bone segmentation using the ray castingtechnique. Medical & Biological Engineering & Computing, December 2011,Volume 49, Issue 12, pp 1413-1424; and Kronman A., Joskowicz L., SosnaJ. (2012) Anatomical Structures Segmentation by Spherical 3D Ray Castingand Gradient Domain Editing. In: Ayache N., Delingette H., Golland P.,Mori K. (eds) Medical Image Computing and Computer-AssistedIntervention-MICCAI 2012. MICCAI 2012. Lecture Notes in ComputerScience, vol 7511. Springer, Berlin, Heidelberg. The contents of thesedocuments are hereby incorporated by reference as if recited in fullherein.

Referring to FIG. 22, the analysis system 10, 10′ performs segmentationof two corresponding implants in CT stacks (block 600). The system 10,10′ derives shape characteristics across the segmentation surfaces(block 604).

Alternatively or additionally, the system 10, 10′ registers focussurface locations that have been pre-defined in relation to implantblueprint (block 606).

The system 10, 10′ can generate unevenly distributed point cloud withhigher concentration at focus surface locations (block 610).

The system 10, 10′ can perform a registration (block 612).

The system 10, 10′ can measure registration accuracy at focus surfacelocations (block 616). The system 10, 10′ can increase point cloudconcentration at locations with inaccurate registration (block 618).

If or when there is sufficient accuracy, the resulting registration canbe provided as input to further processing (block 614).

FIG. 23 illustrates an example of actions for measurement using implantblueprint data for implant analysis systems 10, 10′ according toembodiments of the present invention. The digital implant blueprint canbe utilized to improve measurements. In order to get a high-precisiondefinition of the points on an implant where measurements are to bemade, embodiments of the invention allow them to be identified orselected on the implant blueprint instead of on the implant as depictedin the CT stack. The point definition can then be electronicallytransferred to the image-domain implant through the registrationprocess.

Referring to FIG. 23, the system 10, 10′ can perform segmentation andregistration of two corresponding implants in respective CT stacks(stack pairs) (block 700).

The system 10, 10′ can retrieve measurement reference points that havebeen pre-defined in relation to the implant blueprint (block 702).

The system can retrieve results of previously performed registrations ofimplant blueprint to the respective segmented implant object (block703).

Alternatively or additionally, the software performs registrations ofimplant blueprint to the respective segmented implant object (block706).

The system 10, 10′ translates the blueprint reference points to thesegmented implant objects using the registration (block 708). The systemcomputes CTMA analysis using the translated reference points (block710).

FIG. 24 illustrates example quality assurance actions for batch implantanalysis systems 10′ according to embodiments of the present invention.The system 10′ can be configured to facilitate quality assurance of theanalysis of the entire batch. It can be time-consuming to review thequality of the registration for each comparison in the cohort.Embodiments of the invention can provide an aggregated view of allregistrations. One component can be to visualize all objects of study,overlaid on each other with location relative to their respectivereference object, such that overlapping regions are de-emphasized inorder to accentuate outliers. The de-emphasis can be achieved by makingtransparency or translucency of a sub-region inversely proportional tothe number of objects overlapping in that sub-region.

Another aggregated view can be to represent all the comparisons as itemsin a list, with text and/or thumbnail images, optionally sort themaccording to amount of movement. The list can be linked to theaggregated view described above, such that clicking in one of the viewshighlights the corresponding parts of the other view.

FIGS. 33A-33C are examples of a display 20 with a user interface 30 withan aggregated view V that represents corresponding objects (implantsand/or implants and relevant anatomy) from the different patient imagesand thumbnail images 120 with the implant I for each stack, where eachrow of thumbnail images 120 r is for a respective separate patient atdifferent time points, shown as two different time points. Thisillustration is shown with respect to a very small (atypical) batch ofdifferent patients for discussion purposes. FIG. 33B illustrates that aspecific patient can be selected by a user using the user interface 30.A border 121 can be used to visually correlate the selected patientimage to the corresponding patient's thumbnail images. Selection of animage or images of a patient can be made in both the aggregated view Vand the thumbnail views 120 of a particular patient. For example, asshown in FIG. 33B, a user can select either the thumbnail view(s) of apatient which can prompt the display to visually emphasize thecorresponding implant in the aggregate view V, shown as by providing aperimeter border in color in the aggregated view V or a user can selectan object in the aggregated view V to prompt the system to visuallycorrelate the corresponding patient thumbnail views in the thumbnails120, optionally by generating a color border in the thumbnail view. FIG.33C illustrates that for a selected patient image in the aggregated viewV, other views can be shown such as the registered implant of theselected patient provided in a color-coded view 175 representingregistration quality (such as discussed above with respect to FIGS.17A-J).

Referring to FIG. 24, the system 10′ performs CTMA analysis for thebatch of cases (block 800). The system 10′ generates aggregated view ofthe cases' segmentations and registrations for visual assessment (block810). The system 10′ is configured to display the aggregated views andallow a user to examine the aggregated view (block 812). The system 10′is configured to allow a user to flag a case if the user identifies acase with suspected error (block 815).

If errors are not identified, the system 10′ can be configured to allowa user to enter a verification that the quality of batch analysis hasbeen verified as valid and approved (block 814).

If the user identifies a suspected error, the system 10′ can allow auser to select a view specific for the suspicious case, optionally in anenlarged format relative to the aggregated view, and present that viewto the display 20 to allow the user to determine whether there is anactual error (block 817).

As shown in FIG. 25, the aggregated view “V” comprises a plurality ofdifferent registered/segmented objects from patient implant imagesoverlaid to visually accentuate outliers that may indicate an error Ethat can be electronically selected or separate viewing by a user, suchas by touch screen, mouse or button input.

The system 10′ can allow the user to correct an identified error and thesystem 10′ can then re-run the movement analysis for that patient case(block 818 r ₁) and/or for rerun of all cases (block 818 r ₂).

FIG. 26 is a schematic illustration of an automated image analysissystem 10, 10′ with a display 20, a user interface 30 and an imageprocessing circuit 10 c. The image processing circuit 10 c can includeone or more processors and can be partially or totally held in aworkstation (W) with the display 20 or may be partially or totallyremote from a workstation, such as held in one or more servers 150 andaccessible via a network (i.e., the Internet) via firewalls. The one ormore servers 150 can be integrated into a single server or may bedistributed into one or more servers or other circuits or databases at asingle physical site or at spatially separate locations. Similarly, theimplant movement analysis module 124 can be held by the one or moreservers 150 and can be distributed into multiple processors or databasesor integrated into one or held entirely at a workstation “W”.

The image processing circuit 10 c can be configured to providethumbnails 120 of visualizations of images with targets analyzed acrossstack pairs to the display 20, optionally connected to the userinterface 30, which may be a graphic user interface.

The image processing circuit 10 c can be configured to providethumbnails 130 of images with respective objects from image stack pairsvisually emphasized (highlighted). The image processing circuit 10 c canbe configured to import or select patient images P or batches of imagesP for analysis.

The systems 10, 10′ can be configured to provide defined implantblueprint data 35 (optionally with type, manufacturer, product name,model or the like or a virtual replica image of the implant) that can beselected by a user for analysis in patient images or may be identifiedby metadata or other patient file data.

The server 150 may be embodied as a standalone server or may becontained as part of other computing infrastructures. The server 150 maybe embodied as one or more enterprise, application, personal, pervasiveand/or embedded computer systems that may be standalone orinterconnected by a public and/or private, real and/or virtual, wiredand/or wireless network including the Internet, and may include varioustypes of tangible, non-transitory computer-readable media. The server150 may also communicate with the network via wired or wirelessconnections, and may include various types of tangible, non-transitorycomputer-readable media.

The server 150 can be provided using cloud computing which includes theprovision of computational resources on demand via a computer network.The resources can be embodied as various infrastructure services (e.g.,compute, storage, etc.) as well as applications, databases, fileservices, email, etc. In the traditional model of computing, both dataand software are typically fully contained on the user's computer; incloud computing, the user's computer may contain little software or data(perhaps an operating system and/or web browser) , and may serve aslittle more than a display terminal for processes occurring on a networkof external computers. A cloud computing service (or an aggregation ofmultiple cloud resources) may be generally referred to as the “Cloud”.Cloud storage may include a model of networked computer data storagewhere data is stored on multiple virtual servers, rather than beinghosted on one or more dedicated servers.

Users can communicate with the server 150 via a computer network, suchas one or more of local area networks (LAN), wide area networks (WAN)and can include a private intranet and/or the public Internet (alsoknown as the World Wide Web or “the web” or “the Internet.” The server150 can include and/or be in communication with the implant movementanalysis module 124 using appropriate firewalls for HIPPA or otherregulatory compliance.

Embodiments of the present invention may take the form of an entirelysoftware embodiment or an embodiment combining software and hardwareaspects, all generally referred to herein as a “circuit” or “module.”Furthermore, the present invention may take the form of a computerprogram product on a (non-transient) computer-usable storage mediumhaving computer-usable program code embodied in the medium. Any suitablecomputer readable medium may be utilized including hard disks, CD-ROMs,optical storage devices, a transmission media such as those supportingthe Internet or an intranet, or magnetic storage devices. Some circuits,modules or routines may be written in assembly language or evenmicro-code to enhance performance and/or memory usage. It will befurther appreciated that the functionality of any or all of the programmodules may also be implemented using discrete hardware components, oneor more application specific integrated circuits (ASICs), or aprogrammed digital signal processor or microcontroller. Embodiments ofthe present invention are not limited to a particular programminglanguage.

Computer program code for carrying out operations of data processingsystems, method steps or actions, modules or circuits (or portionsthereof) discussed herein may be written in a high-level programminglanguage, such as Python, Java, AJAX (Asynchronous JavaScript), C,and/or C++, for development convenience. In addition, computer programcode for carrying out operations of exemplary embodiments may also bewritten in other programming languages, such as, but not limited to,interpreted languages. Some modules or routines may be written inassembly language or even micro-code to enhance performance and/ormemory usage. However, embodiments are not limited to a particularprogramming language. As noted above, the functionality of any or all ofthe program modules may also be implemented using discrete hardwarecomponents, one or more application specific integrated circuits(ASICs), or a programmed digital signal processor or microcontroller.The program code may execute entirely on one (e.g., a workstation)computer, partly on one computer, as a stand-alone software package,partly on the workstation's computer and partly on another computer,local and/or remote or entirely on the other local or remote computer.In the latter scenario, the other local or remote computer may beconnected to the user's computer through a local area network (LAN) or awide area network (WAN), or the connection may be made to an externalcomputer (for example, through the Internet using an Internet ServiceProvider).

The present invention is described in part with reference to flowchartillustrations and/or block diagrams of methods, apparatus (systems) andcomputer program products according to embodiments of the invention. Itwill be understood that each block of the flowchart illustrations and/orblock diagrams, and combinations of blocks in the flowchartillustrations and/or block diagrams, can be implemented by computerprogram instructions. These computer program instructions may beprovided to a processor of a general purpose computer, special purposecomputer, or other programmable data processing apparatus to produce amachine, such that the instructions, which execute via the processor ofthe computer or other programmable data processing apparatus, createmeans for implementing the functions/acts specified in the flowchartand/or block diagram block or blocks.

These computer program instructions may also be stored in acomputer-readable memory that can direct a computer or otherprogrammable data processing apparatus to function in a particularmanner, such that the instructions stored in the computer-readablememory produce an article of manufacture including instruction meanswhich implement the function/act specified in the flowchart and/or blockdiagram block or blocks.

The computer program instructions may also be loaded onto a computer orother programmable data processing apparatus to cause a series ofoperational steps to be performed on the computer or other programmableapparatus to produce a computer implemented process such that theinstructions which execute on the computer or other programmableapparatus provide steps for implementing some or all of thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowcharts and block diagrams of certain of the figures hereinillustrate exemplary architecture, functionality, and operation ofpossible implementations of embodiments of the present invention. Inthis regard, each block in the flow charts or block diagrams representsa module, segment, or portion of code, which comprises one or moreexecutable instructions for implementing the specified logicalfunction(s). It should also be noted that in some alternativeimplementations, the functions noted in the blocks may occur out of theorder noted in the figures. For example, two blocks shown in successionmay in fact be executed substantially concurrently or the blocks maysometimes be executed in the reverse order or two or more blocks may becombined, depending upon the functionality involved.

As illustrated in FIG. 27, embodiments of the invention may beconfigured as a data processing system 116, which can include a (one ormore) processors 10 p, a memory 136 and input/output circuits 146. Theone or more processors 10 p can be part of the image processing circuit10 c. The data processing system may be incorporated in, for example,one or more of a personal computer, database, workstation W, server,router or the like. The system 116 can reside on one machine or bedistributed over a plurality of machines. The processor 10 pcommunicates with the memory 136 via an address/data bus 148 andcommunicates with the input/output circuits 146 via an address/data bus149. The input/output circuits 146 can be used to transfer informationbetween the memory (memory and/or storage media) 136 and anothercomputer system or a network using, for example, an Internet protocol(IP) connection. These components may be conventional components such asthose used in many conventional data processing systems, which may beconfigured to operate as described herein.

In particular, the processor 10 p can be commercially available orcustom microprocessor, microcontroller, digital signal processor or thelike. The memory 136 may include any memory devices and/or storage mediacontaining the software and data used to implement the functionalitycircuits or modules used in accordance with embodiments of the presentinvention. The memory 136 can include, but is not limited to, thefollowing types of devices: ROM, PROM, EPROM, EEPROM, flash memory,SRAM, DRAM and magnetic disk. In some embodiments of the presentinvention, the memory 136 may be a content addressable memory (CAM).

As further illustrated in FIG. 27, the memory (and/or storage media) 136may include several categories of software and data used in the dataprocessing system: an operating system 152; application programs 154;input/output device drivers 158; and data 156. As will be appreciated bythose of skill in the art, the operating system 152 may be any operatingsystem suitable for use with a data processing system, such as IBM®,OS/2®, AIX® or zOS® operating systems or Microsoft® Windows®95,Windows98, Windows2000 or WindowsXP operating systems, Unix or Linux™,IBM, OS/2, AIX and zOS are trademarks of International Business MachinesCorporation in the United States, other countries, or both while Linuxis a trademark of Linus Torvalds in the United States, other countries,or both. Microsoft and Windows are trademarks of Microsoft Corporationin the United States, other countries, or both. The input/output devicedrivers 158 typically include software routines accessed through theoperating system 152 by the application programs 154 to communicate withdevices such as the input/output circuits 146 and certain memory 136components. The application programs 154 are illustrative of theprograms that implement the various features of the circuits and modulesaccording to some embodiments of the present invention. Finally, thedata 156 represents the static and dynamic data used by the applicationprograms 154 the operating system 152 the input/output device drivers158 and other software programs that may reside in the memory 136.

The data 156 may include (archived or stored) digital image data sets122 with metadata correlated to respective patients. As furtherillustrated in FIG. 27, according to some embodiments of the presentinvention, the application programs 154 include a movement analysismodule 124. The application programs can also include an implantblueprint module 126. The application program 154 may be located in alocal server (or processor) and/or database or a remote server (orprocessor) and/or database, or combinations of local and remotedatabases and/or servers.

While the present invention is illustrated with reference to theapplication programs 154, and modules 124 and 126 in FIG. 27, as will beappreciated by those of skill in the art, other configurations fallwithin the scope of the present invention. For example, rather thanbeing application programs 154 these circuits and modules may also beincorporated into the operating system 152 or other such logicaldivision of the data processing system. Furthermore, while theapplication programs 124, 126 are illustrated in a single dataprocessing system, as will be appreciated by those of skill in the art,such functionality may be distributed across one or more data processingsystems in, for example, the type of client/server arrangement describedabove. Thus, the present invention should not be construed as limited tothe configurations illustrated in FIG. 27 but may be provided by otherarrangements and/or divisions of functions between data processingsystems. For example, although FIG. 27 is illustrated as having variouscircuits and modules, one or more of these circuits or modules may becombined or separated without departing from the scope of the presentinvention.

FIGS. 28A-28D symbolically illustrate translation of a defined referencepoint Rp on an implant blueprint 35 that can be translated to acorresponding segmentation object 105 in a medical image of a patient.It is noted that the translation process is typically automatic and notvisualized as shown for explanatory purposes. FIG. 28A illustrates areference point Rp added to an implant blueprint 35. FIG. 28Billustrates an implant object 105 of the same model as the implantblueprint segmented in a CT stack. FIG. 28C illustrates the implantblueprint 35 and segmentation object 105 are registered. FIG. 28Dillustrates that due to the registration, the blueprint reference pointRp is connected to a specific point on the segmented implant 105.

FIGS. 29A-29C illustrate example point cloud generation. While shownwith respect to an implant, the point cloud generation techniques can beused for other segmented objects such as bones or a combination oftarget bone and implant objects. FIG. 29A illustrates that a point cloudPc can be generated to spatially represent an implant, withsubstantially uniform distribution Pu across the sure external surfaces.That is, the substantially uniform distribution can place points of thepoint clouds with similar distances between all neighboring pointstypically within 20% of the same distance between all each neighboringpoint. FIG. 29B illustrates that focus surface locations Fs can bedefined on an implant blueprint 35 (left side) and a point cloud Pc canbe placed only at those defined focus surface locations (right side).FIG. 29C illustrate that a point cloud can be generated to spatiallyrepresent an implant with higher point concentrations Ph where there isa high curvature of the implant shape relative to surfaces with lowercurvatures.

FIGS. 30A-3C illustrate example templates T for measurement targets. Thereference points Rp for measurement correspond to the objects used asregistration targets. FIG. 30A illustrates a template T with ameasurement target Rp for hip implant movement where the point tomeasure is the tip of the implant stem and movement is relative toreference of the adjacent femur bone B_(f). FIG. 30B illustrates anothertemplate T with another measurement target for hip implant movement,where the point of measure Rp is the point at the top of the cup of theimplant and movement is relative to the reference of the pelvic boneB_(p). FIG. 30C illustrates another template T with a measurement targetfor spinal implant movement. The point of measure Rp is the tip of thelowest screw S₁ and the movement is relative to the reference of thesecond lowest (adjacent/neighboring) screw S₂.

FIGS. 31A-31E illustrate a target template collection Tc of defined(typically pre-defined or default) templates T for measurement andregistration targets that can be used as inputs to automatically performmovement analysis. The measurement points P (red or solid dark circlefor non-color versions), the measurement reference 900 is shown in greenor in cross-hatch shading for non-color versions), and theregistration-only targets 910 are shown in blue (slanted markings innon-color versions).

FIGS. 31A and 31B illustrate templates T with respective measurementtargets 900. FIG. 31A illustrates the implant stem tip P versus thefemur 900. FIG. 31B shows the top of the implant cup P and the pelvis900. FIGS. 31C-31E illustrate example templates T for registration only(for qualitative analysis, no measurements). FIG. 31C shows the fullimplant as the registration target 910. FIG. 31D shows the cup of theimplant as the registration target 910. FIG. 31E shows the stem of theimplant as the registration target 910.

FIGS. 35A-35C illustrate examples of measurement references foranalyzing movement. Movement measurements are reported in terms oftranslation and rotation in 3-dimensional space, in a coordinate systemC, shown as a cartesian X, Y, Z coordinate system. This coordinatesystem C, and therefore all measurements reported therein, are usuallydefined either relative to the patient anatomy A (FIG. 35B), or relativeto the implant I (FIG. 35A). For both, right side or left side variationexists. For measurements to be optimally useful, the coordinate system Cis defined in an as repeatable and precise fashion as possible.

Any point of any implant or tissue with enough radiodensity can be usedas reference points for measurements and/or coordinate systems. This isdone by placing reference points Rp at points of particular interest.Reference points “Rp” can be selected such that the movement analysis issensitive to specific implant failure modes or migration patterns ofinterest. Thus, different implants can have different defined referencepoints Rp. For example, a given implant type might be known to have itsfront moving whereas the part to the back usually has little migrationso more reference points Rp may be provided on the front and less ornone on the back. FIG. 35C illustrates three reference points defined onthe implant, R_(A), R_(B), R_(C) with R_(A) at a front, R_(B) at a topand R_(C) at a back of the implant I.

The foregoing is illustrative of the present invention and is not to beconstrued as limiting thereof. Although a few exemplary embodiments ofthis invention have been described, those skilled in the art willreadily appreciate that many modifications are possible in the exemplaryembodiments without materially departing from the novel teachings andadvantages of this invention. Accordingly, all such modifications areintended to be included within the scope of this invention as defined inthe claims. The invention is defined by the following claims, withequivalents of the claims to be included therein.

That which is claimed:
 1. An automated implant analysis methodcomprising: obtaining a batch of image data sets of a plurality ofdifferent patients having an implant coupled to bone; providing a firstdata set of a first patient from the batch of image data sets, the firstdata set comprising a first image stack and a second image stack;allowing a user to select parameter settings for implant movementanalysis of the implant including selecting a first object of interestand a second reference object; segmenting the first image stack and thesecond image stack to identify corresponding object pairs of the firstobject and the second object; registering each of the identified objectpairs; automatically calculating measurements of movement of the implantand/or coupled bone after the registration; automatically propagatingthe selected parameter settings to other image data sets of otherpatients of the batch of image data sets; and electronicallyautomatically repeating the segmentation, registration and calculatedmeasurements for the batch of image data sets of others of the differentpatients.
 2. The method of claim 1, wherein the first object is a targetstudy object, wherein one of the first object and the second referenceobject is the implant, wherein the parameter settings include relating acoordinate system to the reference object, and identifying whichmeasurements are to be calculated such as rotation and location ofselected points of interest of the target study object.
 3. The method ofclaim 1, wherein, after the first data set is analyzed, for identifyingthe first and second objects in the image data sets of the others of thedifferent patients before a respective registration, the implant andassociated position is first automatically electronically identified,then the second object is identified using the implant position asguidance.
 4. The method of claim 1, further comprising automaticallyelectronically defining a cohort analysis template based on the userselected parameter settings and the first object and the secondreference object of the data set of the first patient, and wherein thecohort analysis template is used to automatically propagate the selectedparameter settings to the other image data sets thereby using identicalparameter settings across all comparisons provided by the calculatedmeasurements.
 5. The method of claim 1, further comprising providing adisplay of results of the calculated measurement of movement of theimplant in the batch of image data sets.
 6. The method of claim 1,further comprising providing a visualization of an aggregated view ofoverlying registered images of image data sets of the different patientswith overlapping regions visually deemphasized relative to outliers. 7.The method of claim 6, wherein the overlapping regions have a reducedoptical opacity relative to the outliers and/or are presentedtranslucent or transparent.
 8. The method of claim 6, wherein thevisualization is presented with sub-regions shown with differentopacities or contrast, and wherein different sub-regions are shown withan opacity and/or contrast that is inversely proportional to a number ofobjects overlapping in a respective sub-region.
 9. The method of claim1, further comprising displaying thumbnail images of registered objectsof different patients, optionally sorted by amount of calculatedmeasurement of movement.
 10. The method of claim 9, further comprising:electronically linking the thumbnail images to an aggregated view of allthe registered objects of the different patients; and allowing a user tonavigate from a selected thumbnail image to the aggregated view,optionally with the selected thumbnail image visually emphasized in theaggregated view relative to other registered images of other thumbnailimages.
 11. The method of claim 1, wherein the segmenting step iscarried out automatically, and wherein the method further comprisesautomatically repeating the segmenting step using different tuningparameters before the registering step to thereby provide more accuratesegmentation of the first and second objects.
 12. The method of claim 1,further comprising, before the segmenting step, automatically selectingrelevant image stack pairs from the first and second patient imagestacks, wherein the image stack pairs have the first and/or secondobject.
 13. The method of claim 1, further comprising providing anelectronic implant blueprint corresponding to the implant, wherein oneor more of the segmenting, registering or calculating measurements iscarried out using the electronic implant blueprint.
 14. The method ofclaim 1, further comprising providing an electronic implant blueprintcorresponding to the implant, wherein the segmenting is carried out aplurality of times for the first data set using a plurality of differentthreshold levels that varies noise levels to match the blueprint withthe segmented first and/or second object.
 15. The method of claim 1,further comprising providing an electronic implant blueprintcorresponding to the implant, wherein the registration comprisesmatching point clouds of points generated on one or more surfaces of thefirst and/or second object.
 16. The method of claim 1, furthercomprising: providing an electronic implant blueprint corresponding tothe implant; defining points on the electronic implant blueprint wheremeasurements are to be made; and transferring the defined points to animage-domain implant, wherein the registration is carried out using thedefined points.
 17. The method of claim 1, further comprising: providingan electronic implant blueprint corresponding to the implant;electronically defining reference points on the electronic implantblueprint; then electronically translating the blueprint referencepoints to the segmented implant object, wherein the automaticallycalculating measurements of movement of the implant and/or coupled boneafter the registration is carried out using the translated blueprintreference points.
 18. The method of claim 1, further comprising:providing an electronic implant blueprint corresponding to the implant;and electronically defining focus surface locations on the electronicimplant blueprint, wherein, before the registration, automaticallyelectronically translating the blueprint focus surface locations tocorresponding locations on segmented first and/or second object, thengenerating an unevenly distributed point cloud with higher concentrationat focus surface locations, then performing the registration using thegenerated point cloud.
 19. The method of claim 1, wherein, before theregistration, automatically electronically deriving shapecharacteristics across one or more surfaces of a segmented first and/orsecond object, then electronically defining high curvature locations asfocus surface locations, then electronically generating an unevenlydistributed point cloud with higher concentration at focus surfacelocations, then electronically performing the registration using thegenerated point cloud.
 20. The method of claim 1, further comprising:providing an electronic implant blueprint corresponding to the implantin the first patient; electronically comparing a segmented first orsecond reference object to the implant blueprint; and adjustingsegmentation parameters and repeating the segmentation of the first dataset.
 21. A workstation comprising or in communication with an imageprocessing circuit configured to carry out the method of claim 1.